The new features of AbiPy v0.9.1

M. Giantomassi and the AbiPy group

10th international ABINIT developer workshop
May 31 - June 4, 2021 - Smart Working, Lockdown@BE

---

• These slides have been generated using jupyter, nbconvert and revealjs
• The notebook can be downloaded from this github repo
• To install and configure the software, follow these installation instructions

Use the Space key to navigate through all slides.

What is AbiPy?

Python package for:

• Generating ABINIT input files automatically
• Post-processing results extracted from netcdf or text files
• Interfacing ABINIT with external tools such as vesta, wannier90, lobster, etc.
• Running ABINIT-specific workflows on laptops as well as on HPC clusters

Dependencies:

• Hard deps: pymatgen, netcdf4, matplotlib, plotly, numpy, scipy, pandas, ipython …
• Soft deps: ASE, phonopy, jupyter …

NB: AbiPy can be interfaced with other packages (e.g ASE, phonopy) via converters.

How to install AbiPy

Using conda and the conda forge channel (recommended):

    conda install abipy --channel conda-forge

Since conda is not limited to python packages, one can install ABINIT in the same env with:

    conda install abinit -c conda-forge

NB: conda packages for AbiPy and ABINIT are now provided by conda-forge therefore:

• new pkgs are automatically built when new releases are pushed to github
• the abiconda channel is deprecated. Please use conda-forge.

---

Using pip and python wheels:

    pip install abipy --user

---

For further info see http://abinit.github.io/abipy/installation.html

What's new at the level of the documentation?

• New website based on sphinx-rtd-theme
• More examples:
  • Gallery of plots based on matplotlib and plotly (59 scripts)
  • New examples for Flows (38 scripts)
  • Each example can be executed inside a Docker container on mybinder.org. Just click the LaunchBinder badge.
• New notebook tutorial for elastic properties with AbiPy
• Additional examples for post-processing tools in the ABINIT EPH tutorials

#%embed https://abinit.github.io/abipy/gallery/index.html
#%embed https://abinit.github.io/abipy/flow_gallery/index.html

What's new at the level of workflow infrastructure?

In addition to workflows/tools for GS, DFPT, $GW$, IPA optics, BSE, we now have:

• Python converters: DDB $\;\rightleftarrows \;$ phonopy / tdep

• New Flows and improved post-processing tools for:

  • elastic and piezoelectric tensors with DFPT (clamped/relaxed atoms)
  • non-linear optical properties (SHG) with DFPT
  • Phonopy with AbiPy flows
  • Gruneisen parameters with DFPT + finite differences
  • effective masses with DFPT or finite differences
  • e-ph self-energy, e-ph matrix elements and scattering potentials
  • phonon-lmited transport properties

DDB converters

(contributed by G. Petretto)

• Based on previous work by H. Xue, E. Bousquet and A. Romero

• Can be used to:

  • connect the ABINT DPT part with other packages requiring phonopy files (e.g. anharmonic calculations with hipive)
  • interface AbiPy with phonopy tools e.g. irreps

• Algorithm:

  • Runs anaddb to get the interatomic force constants IFC($\bf{R}$), BECs and $\epsilon^\infty$
  • Save results in nc format (anaddb.nc)
  • Convert IFC($\bf{R}$) and tensors from ABINIT to phonopy conventions
  • Optionally, save phonopy files in output_dir_path.

In python, everything boils down to:

ddb = abilab.abiopen("mp-149_DDB")
phonopy_obj = ddb.anaget_phonopy_ifc(output_dir_path="output_dir")

• Example: obtain phonon irreps from DDB using phonopy

ddb = abilab.abiopen("mp-149_DDB")
ph = ddb.anaget_phonopy_ifc(ngqpt=[1,1,1])
ph.set_irreps([0, 0, 0])
ph.get_irreps().show()

Overview of the new Flows

⚠️ There are two different workflow infrastructures:

Internal AbiPy implementation (abipy.flowtk modules):

• ✅ Lightweigth, no database required
• ✅ Designed for rapid prototyping and/or for supporting advanced ABINIT capabilities
• ❌ No explicit support for high-throughput (HT) applications.

AbiFlows package (requires Fireworks and MongoDB database)

• ✅ HT-oriented: use MongoDB to store workflow status and results for further analysis
• ✅ High-level API designed for HT applications (e.g. phonon calculations for the materials project)
• ❌ Not all the flowtk workflows are immediately available in AbiFlows.

NB: There's an ongoing effort to reimplement AbiFlows in terms of the atomate framework. In this talk, we will mainly discuss the new features available in abipy.flowtk.

Effective masses with DFPT

This flow performs:

• SCF followed by NSCF run along k-path to find band edges automatically
• Compute $\epsilon^{\alpha\beta}_{n\bf{k}}$ and the effective mass tensor for these k-points from $|u_{n\mathbf{k}}\rangle$, and the k-derivatives $|u_{n\mathbf{k}}^\alpha\rangle$ $H^\alpha_{\mathbf{k}}$, $H^{\alpha\beta}_{\mathbf{k}}$

For the formalism, see J. Laflamme Janssen, et. al. Phys. Rev. B 93, 205147

Python API:

# Build input for GS SCF calculation.
scf_input = make_scf_input()   #

from abipy.flowtk.effmass_works import EffMassAutoDFPTWork
flow = flowtk.Flow("effmass_flow")

work = EffMassAutoDFPTWork.from_scf_input(scf_input)
flow.register_work(work)

Dependency Graph

---

• run_effmass_dfpt.py --sheduler
• Submission with optimized MPI-params.
• Support fot bluegene, moab, pbspro, sge, shell, slurm, torque
• Restart capabilities (e.g. timeout limit) and error handlers
• Indipendent tasks are executed in parallel

To generate a GS input, one can use factory functions (HT-oriented)

from abipy.abio.factories import gs_input

gs_input(structure="si.cif", pseudos="14si.pspnc", ecut=8)

##############################################
#### SECTION: basic
##############################################
ecut 8
ngkpt 8 8 8
shiftk 0.5 0.5 0.5
nshiftk 1
kptopt 1
nsppol 2
nband 16
occopt 3
tolvrs 1e-08
##############################################
#### SECTION: gstate
##############################################
spinat
0.0 0.0 0.6
0.0 0.0 0.6
chksymbreak 0
nspinor 1
nspden 2
charge 0.0
tsmear 0.0036749322175655 Ha
##############################################
#### STRUCTURE
##############################################
natom 2
ntypat 1
typat 1 1
znucl 14
xred
0.0000000000 0.0000000000 0.0000000000
0.2500000000 0.2500000000 0.2500000000
acell 1.0 1.0 1.0
rprim
6.3285005244 0.0000000000 3.6537614813
2.1095001748 5.9665675141 3.6537614813
0.0000000000 0.0000000000 7.3075229627

or build it from scratch if full controll is wanted:

other_inp = abilab.AbinitInput(structure="si.cif", pseudos="14si.pspnc")

other_inp.set_vars(ecut=8, nsppol=2)
other_inp.set_autokmesh(8)

other_inp

##############################################
#### SECTION: basic
##############################################
ecut 8
nsppol 2
ngkpt 8 8 8
kptopt 1
nshiftk 4
shiftk
0.5 0.5 0.5
0.5 0.0 0.0
0.0 0.5 0.0
0.0 0.0 0.5
##############################################
#### STRUCTURE
##############################################
natom 2
ntypat 1
typat 1 1
znucl 14
xred
0.0000000000 0.0000000000 0.0000000000
0.2500000000 0.2500000000 0.2500000000
acell 1.0 1.0 1.0
rprim
6.3285005244 0.0000000000 3.6537614813
2.1095001748 5.9665675141 3.6537614813
0.0000000000 0.0000000000 7.3075229627

Other AbiPy applications are presented in the following talks:

• Electron-phonon beyond Fröhlich: dynamical quadrupoles in polar and covalent solids by G. Brunin
• Phonon-limited conductivity in 2D and 3D metals by O. Nadeau
• Absorption spectrum calculations using cumulant expansion in electron-phonon interactions by J. C. Abreu
• Automating ΔSCF computations of point defects using AbiPy workflows by J. Bouquiaux

What's new at the level of the post-processing tools?

• New plotting tools based on plotly
• GUIs and dashboards based on panel and bokeh

---

Why plotly?

• ✅ Publication quality figures.
• ✅ Flexible python API able to produce rather advanced plots
• ❌ Plots are difficult to customize without changing the python code
• ❌ Plots lacks interactivity and integration with HTML/JS

• ✅ Interactive plots + chart editor GUI to customize the figure
• ✅ Plays well with HTML (plotly is written in js with python bindings)
• ❌ Open source project but not all the features are available in the free plan
• ❌ Requires browser (this may represent an issue on some HPC centers)

AbiPy plots with matplotlib

gsr = abiopen("si_nscf_GSR.nc")
gsr.ebands.plot(with_gaps=True);

AbiPy plots with plotly

(contributed by Y. He)

gsr.ebands.plotly(with_gaps=True);  # obj.plot becomes obj.plotly

To update the plotly figure to the chart studio server, use:

gsr.ebands.plotly(with_gaps=True, chart_studio=True);

---

Users can finally customize the AbiPy plot without changing the python code 🎉

Interactive 3d plots with plotly:

gsr.ebands.kpoints.plotly(title="k-path in 3d with plotly");

And, obviously, phonons with plotly:

# Open the PHBST file produced by anaddb and get the phonon bands.
with abiopen(abidata.ref_file("trf2_5.out_PHBST.nc")) as ncfile:
    phbands = ncfile.phbands

# Read the Phonon DOS from the netcd file produced by anaddb (prtdos 2)
with abiopen(abidata.ref_file("trf2_5.out_PHDOS.nc")) as ncfile:
    phdos = ncfile.phdos

phbands.plotly_with_phdos(phdos, units="cm-1", title="AlAs Phonon bands + DOS in cm-1");

Warning: file /Users/gmatteo/git_repos/abipy/abipy/data/refs/alas_phonons/trf2_5.out_PHBST.nc does not contain atomic_numbers.
Particular methods need them!

Integrating AbiPy with web-based technologies via Panel

• AbiPy GUIs inside jupyter notebooks
• Dashboards and web apps
• Integration with the AbiPy command line interface
• Web apps for ABINIT users (🚧)

Pros and cons of the client-server model

Advantages:

• The client does not need to install the scientific software stack (when running on different machines)
• Can implement web apps that allows the user to upload data and analyze the results e.g DDB GUI.

Disavantages

• Round trip delay if client != host and slow connection
• Upoloadng a 1Gb file to the remote server just because you don't want to install software on the localhost is a very bad idea.
• OK for relatively small files (< 1Gb) but this approach is not designed to handle big data.
• Not all the HPC centers provide specialized nodes to post-process the results inside a web browser/notebook.

#gsr = abiopen("si_nscf_GSR.nc")
#abilab.abipanel();
#gsr.get_panel()

How to use AbiPy panels inside jupyter notebooks

To build a panel GUI inside the notebook, call the get_panel method:

ddb = abilab.abiopen("ZnSe_hex_qpt_DDB")

abilab.abipanel(); # Important
ddb.get_panel()

Don't be surprised if you start to click buttons and nothing happens in the GUI! One needs a running python backend to execute the callbacks triggered by the widgets.

---

To open a dashboard for the DDB file from the shell, use:

abiopen.py out_DDB --panel  # or -pn

Creating dashboards from the command line

• Creating panels inside a notebook is great if you needs GUIs plus the possibility of executing python code.
• There are however cases in which we only need a dashboard with widgets to interact with the data.

To open a dashboard for the DDB file from the shell, use the --panel option:

abiopen.py out_DDB --panel  # or -pn if you prefer the short option.

---

To open a dashboard for the DDB file from the shell, use the --panel option:

Documentation for these new features available at:

%embed https://abinit.github.io/abipy/graphical_interface.html

#plotter = abilab.ElectronBandsPlotter()
#plotter.add_ebands(label="BZ sampling", bands="si_scf_GSR.nc")
#plotter.add_ebands(label="k-path", bands="si_nscf_GSR.nc")
#plotter.gridplot(with_gaps=True);

How to run AbiPy Flows

In order to run a Flow, we need two configurations files:

1. scheduler.yml providing:

   • scheduler parameters such as days, hours, minutes, max_njobs_inqueue, max_ncores_used, …

1. manager.yml providing:

   • list of shell commands to be executed before running ABINIT
   • list of modules to load
   • options for the queue manager (bluegene, moab, pbspro, sge, shell, slurm, torque)

See this page for examples or use the abidoc.py script and the syntax:

• abidoc.py scheduler
• abidoc.py manager
• abidoc.py manager slurm

How to run calculations?

• The simplest way to start the scheduler from the shell is via the syntax:

run_elastic.py --scheduler # -s for the short option

• For non-trivial Flows, we suggest to put the scheduler in background and use nohup so that we can disconnect from the shell session without killing the scheduler.

nohup run_elastic.py --s > log 2> err &

• Obviously, it is also possible to submit a Slurm script to execute the script on the compute note with 1 core.